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I.  INTRODUCTION 


A  one-dimensional  random  walk  (r.w.)  is  a  sequence,  {Zn},  whose  terms  are  given  by 


Zn  =  £  uK.  (1.D 

K  =  1 

where  {Ur}  is  a  random  sequence  of  real  numbers.  A  segment  of  the  random  walk  (Zn)  is  simply  a 
set  of  the  form,  { Zn}  consisting  of  (N2  _  Nx  +1)  consecutive  terms  of  { Zn} .  In  this  report 
we  will  confine  our  attention  to  r.w.’s  whose  underlying  random  sequences,  {UK},  are  stationary, 
gaussian,  and  zero  mean. 

The  usual  interpretation  of  a  one-dimensional  r.w.  may  be  stated  as  follows.  Suppose  that  a 
particle  is  constrained  to  move  along  the  x-axis.  If  the  particle  starts  at  the  origin  and  at  time 
tK  =  Kh,  K  =  1,  2,  . . . ,  moves  |  UK  |  units  to  the  right  if  UK  is  positive  or  to  the  left  if  UK  is  nega¬ 
tive,  then  Zn  represents  the  position  of  the  particle  at  time  tn.  The  constant,  h,  denotes  the  time 
interval  between  steps.  Thus,  using  this  interpretation,  we  may  think  of  {UK}  and  {Zn}  as  time 
sequences;  i.e.,  let  UK  =  U(tK)  and  Zn  =  Z(tn). 

In  many  applied  problems  that  involve  r.w.’s,  it  is  necessary  to  model  segments  of  an  r.w.  as 
simple  algebraic  functions.  Assuming  that  Zn  =  Z(tn),  polynomials  of  the  form, 

d 

Pd(t)  =  Z]  ajtJ’  d  =  degree  of  Pd(t) ,  (1.2) 

i=o 


are  particularly  convenient  models  for  various  applications.  Given  an  r.w.  segment,  a  polynomial 
of  degree  d  may  be  fit  to  the  points  of  the  segment,  using  some  fitting  criterion.  Throughout  this 
report,  we  will  use  the  least-squares  (l.s.)  criterion  for  all  polynomial  fits;  i.e.,  the  coefficients  of 
the  polynomial  will  be  determined  such  that  the  sum  of  the  squares  of  the  fit  residuals  is  minimized. 
The  fit  residuals,  r;,  are  simply  the  differences  between  the  points  of  the  segment  and  the  values  of 
the  fit  polynomial  at  corresponding  times.  We  will  take  these  residuals  in  the  sense, 

r;  =  Zj-Z,,  (1.3) 


where  Z-1  =  Pd(t,). 

It  is  important  to  note  that  by  fitting  (in  the  least-squares  sense)  a  polynomial  to  an  r.w. 
segment  we  are  not  performing  linear  estimation.  Instead,  we  are  performing  linear  approximation. 
When  you  perform  linear  l.s.  estimation  to  determine  the  coefficients  of  a  fitting  polynomial  on  the 
basis  of  a  particular  set  of  data,  you  make  two  assumptions.  First,  you  assume  that  the  data  you 
are  given  consist  of  observations  that  have  been  corrupted  by  noise.  Second,  you  implicitly  assume 
that,  if  the  noise  on  the  data  was  not  present,  then  the  estimation  algorithm  would  yield  a  perfect 
fit;  i.e.,  that  the  polynomial  is  a  perfect  model  for  the  process  that  generated  the  basis(uncorrupted) 
data.  Thus,  the  coefficients  determined  by  the  l.s.  estimation  algorithm  will  be  in  error  due  to  the 
effects  of  the  data  noise,  in  the  sense  that  they  will  differ  from  the  exact  values  they  would  have  if 
no  data  noise  was  present. 
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In  contrast,  when  you  perform  linear  approximation  using  the  l.s.  criterion  you  make  two 
very  different  assumptions.  First,  you  assume  that  the  given  data  set  has  not  been  corrupted  by 
noise.  Second,  you  assume  that  the  form  of  the  fitting  polynomial  is  not  equivalent  to  the  process 
that  generated  the  data  but  is  simply  an  approximation  to  that  process.  Thus,  the  coefficients 
determined  by  l.s.  approximation  are  not  in  error;  they  are  correct  in  the  sense  that  any  other 
polynomial  of  the  same  degree  would  yield  a  larger  sum  of  the  squares  of  the  residuals,  taken  with 
respect  to  the  given  data  set.  It  follows  that  it  is  not  meaningful  to  talk  about  the  statistics  of 
errors  in  fit  coefficients,  as  we  would  if  l.s.  estimation  was  being  performed. 

We  may,  however,  ask  meaningful  statistical  questions  about  l.s.  polynomial  fits  to  r.w. 
segments,  even  though  the  coefficients  are  exact  in  the  sense  explained  in  the  preceding  paragraph. 
For  example,  what  are  the  statistics  of  the  fit  residuals  for  a  l.s.  polynomial  fit  of  degree  d  to  an 
r.w.  segment  of  length  m?  If  more  than  one  segment  of  a  given  r.w.  has  been  approximated,  how 
are  the  residuals  of  the  fits  to  the  different  segments  related  statistically?  For  an  arbitrary  segment 
of  length  m  of  a  given  r.w.,  what  is  the  variance  of  each  of  the  coefficients  in  the  fit  polynomial? 
These,  and  other  questions  to  be  discussed  later,  are  well  posed  and  have  definite  answers  once  the 
statistics  of  the  underlying  random  sequence,  {UK},  have  been  specified. 

The  primary  purpose  of  this  report  is  to  answer  the  three  questions  just  mentioned.  We 
begin  with  a  brief  discussion  of  one-dimensional  r.w.’s  whose  underlying  random  sequences,  {UK}, 
are  stationary,  gaussian,  and  zero  mean.  The  special  case  of  first  degree  l.s.  polynomial  fits  to  r.w. 
segments  is  then  examined  in  some  detail.  Next,  the  machinery  needed  to  treat  the  general  case  of 
fits  of  arbitrary  degree  is  developed.  Finally,  we  present  a  nontrivial  example  of  our  main  results 
by  calculating  the  statistics  of  the  fit  residuals  when  the  given  r.w.  is  generated  by  the  fractional 
frequency  errors  of  an  atomic  clock. 


II.  ONE-DIMENSIONAL  (GAUSSIAN)  RANDOM  WALKS 

One-dimensional  random  walks  (as  defined  by  Equation  (1.1))  arise  in  many  areas  of  both 
the  “hard”  and  “soft”  sciences.  Fluctuations  in  stock  market  indices,  the  size  of  biotic  populations, 
the  fortunes  of  a  gambler,  and  the  errors  in  a  clock  are  common  examples.  The  motivation  for  the 
work  we  are  reporting  on  comes  from  the  need  to  model  segments  of  the  r.w.’s  that  occur  in  some 
physical  systems  because  of  random  errors  at  the  input  of  a  summing  subsystem.  These  random 
errors  are  often  composites,  composed  of  errors  generated  by  several  separate  error  mechanisms 
such  as  thermal  noise,  stiction  (i.e.,  stick-and-slip  resulting  from  nonuniform  friction  in  mechanical 
assemblies),  and  random  environmental  influences  on  sensitive  components.  When  digital  processing 
is  included  in  the  system,  roundoff  and  truncation  are  also  sources  of  error  within  the  system. 

It  was  mentioned  in  section  I  that  we  will  restrict  our  attention  to  r.w.’s  whose  underlying 
random  sequences,  {UK},  are  gaussian,  stationary,  and  zero  mean.  In  many  physical  systems,  the 
composite  errors  that  exist  at  the  input  of  the  summing  subsystem  may  be  assumed  to  be  gaussian. 
The  mathematical  basis  for  this  assumption  is  a  remarkable  theorem  known  as  the  central  limit 
theorem.  For  our  discussion,  a  useful  form  of  this  theorem  is  (from  Reference  1): 


Theorem.  “Let  xlt  x2,  x3,  xm  be  a  series  of  statistically  independent  random  variables 
having  arbitrary  distributions  for  which  the  means  p1;  p2,  •••>  Pm  and  variances  of,  o\, 
exist.  If  m  is  sufficiently  large,  the  sum  E™  x  xs  will  be  approximately  normally  dis¬ 
tributed  [i.e.,  gaussian]  with  mean  p  and  variance  o2,  whether  the  X;’s  are  normally  distributed 
[gaussian]  or  not  and  where  the  mean  and  variance  of  the  sum  are  given  by  p  =  p;  and 
o2  =2“x  of ,  respectively.” 

It  follows  that  if  each  UK  (interpreted  as  the  input  to  a  summing  subsystem  at  time  tK)  is  composed 
of  a  large  number  of  independent  random  errors,  x^^),  x2(tK),  ...,  xm(tK),  then,  for  large  m,  we 
are  justified  in  assuming  that  each  UK  is  gaussian.  (In  fact,  m  may  not  need  to  be  very  large  at  all 
before  the  distribution  of  UK  becomes  essentially  gaussian.  An  excellent  example  is  given  by 
Papoulis  on  pages  267  and  268  of  Reference  2.)  The  assumption  that  {UK}  is  a  stationary  sequence 
is  based  on  a  requirement  that  the  systems  under  consideration  be  in  steady  state. 

We  are  fortunate  that  composite  errors  in  many  systems  may  be  assumed  to  be  gaussian 
because  gaussian  random  variables  have  several  properties  that  simplify  any  analysis  involving  them. 
For  example,  a  gaussian  distribution  is  completely  specified  by  its  first  two  moments  (i.e.,  by  its 
mean  and  variance).  This  is  not  the  case  for  arbitrary  distributions.  Also,  a  linear  transformation 
of  a  set  of  gaussian  random  variables  yields  a  set  of  gaussian  random  variables.  This  is  clearly  not 
true  for  sets  of  random  variables  with  arbitrary  distributions,  as  evident  from  the  central  limit 
theorem.  (Demonstrations  of  the  truth  of  these  two  statements  concerning  gaussian  random 
variables  may  be  found  in  Reference  2.) 

The  final  restriction  concerning  {UK}  that  holds  throughout  this  report  is  that  it  be  a  zero 
mean  sequence.  This  restriction  guarantees  that  the  terms  of  {Zn}  (as  defined  by  Equation  (1.1)) 
represent  random  sums.  If  underlying  sequences  with  nonzero  means  were  allowed,  the  resulting 
r.w.’s  would  be  composed  of  terms  of  the  form, 


Z 


n 


+  nb, 


where  { WK}  is  a  zero  mean  sequence  and  b  is  a  constant  bias.  Clearly,  the  ramp  sequence,  {nb} , 
may  be  approximated  exactly  by  a  polynomial  and  so  is  of  no  interest  in  this  report. 

Now,  let’s  examine  the  r.w.’s  whose  underlying  sequences  satisfy  our  three  restrictions.  We 
will  look  at  two  cases. 


Case  (1):  {UK}  Uncorrelated  (White) 

Let  {UK}  be  a  white,  stationary,  zero  mean  gaussian  random  sequence,  with  variance  of]. 
Then,  the  r.w.  defined  by 


K  =  1 


u. 


is  a  zero  mean  gaussian  sequence,  and  for  this  case  the  variance  of  its  nth  term  is  given  by 
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al  =  E[Z*]  =  nol,  (2.1) 

where  E[  ]  denotes  the  expectation  operator.  The  correlation  between  Zj  and  ZJ(  I  <  J,  is  given 
by  the  correlation  coefficient. 


Pz(I.J)  =  ®  [ZIZJ]/a1aJ  =  (I/J)1/2. 


In  particular,  if  J  =  I  +  q, 


Pz(L  I  +  d) 


(2.2) 


(2.3) 


(Of  course,  pz(J,  I)  =  pz(I,  J).) 

*  *  * 

Case  (2):  {UK}  Correlated 

Let  {UK}  be  a  stationary,  zero  mean  gaussian  random  sequence  with  variance  a^,  and  with 
the  correlation  between  UK  and  UK+q  given  by  the  correlation  coefficient,  Py(q)  #=  0.  Then,  the 
r.w.  defined  by 

z„  -  £  Uk 

K  =  1 


is  a  zero  mean  gaussian  sequence,  and  for  this  case  the  variance  of  its  nth  term  is  given  by 


°z(l)  =  au 

n  - 1 

N 

> 

4(«)  =  E[Z2]  = 

n  +  2  £  ipu(n-i) 
i  =  l 

,  n  >  2. 

J 

The  correlation  between  Zl  and  Zj  is  given  by  the  correlation  coefficient,  (I  <  J), 


PZ(U)  _  E[ZjZj]/az(I)az(J)  -  —  —  (2.5) 

(I  +  S,)1/2(J  +  Sj)1/2 

where 


l  -l 

Si  =  2  ‘Pud  "  '), 

i  =  1 
J  -1 

Sj  =  2  iPi;(J  —  i), 

i  =  1 

I  J 

S[J  =  V  ^  ptr(j  -  ]) 
i-1  j  =  I  + 1 
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Notice  that  if  pv(q)  =  0  for  q  =  1,  2,  then  Equations  (2.4)  and  (2.5)  reduce  to  Equations  (2.1 ) 
and  (2.2),  respectively,  as  we  would  expect.  Also,  pz(J,  I)  =  pz(I,  J),  as  in  case  ( 1). 

*  *  * 

It  should  be  emphasized  that  when  we  compute  the  expectation  of  a  function  of  terms  from 
an  r.w.,  the  implied  average  is  taken  over  the  ensemble  of  all  possible  r.w.’s  whose  underlying 
sequences  have  the  specified  statistics.  For  example,  the  variance  of  the  nth  term  of  {Zn}  in 
case  (1)  is  interpreted  to  be  the  “average”  value  of  the  squares  of  the  nth  terms  from  all  possible 
r.w.’s  whose  underlying  sequences  are  white,  stationary,  zero  mean,  gaussian  and  have  variance 
ofj.  In  both  case  ( 1)  and  case  (2),  it  is  clear  that  f  Zn}  is  a  nonstationary  sequence. 

In  Figure  2  and  Figure  3  we  have  examples  of  the  first  hundred  steps  of  an  uncorrelated  r.w. 
and  a  correlated  r.w.,  respectively.  These  examples  were  prepared  as  follows.  The  first  hundred 
steps  of  a  white,  stationary,  zero  mean  gaussian  sequence  with  unit  variance  were  chosen  from 
table  XXIII  of  Reference  1.  These  numbers  will  be  denoted  by  {Uk}k°=1’  and  are  plotted  in 
Figure  la.  Next,  the  first  hundred  terms  of  a  stationary,  zero  mean,  first-order  gauss-markov 
sequence,  with  unit  variance  were  generated.  The  correlation  time  of  this  gauss-markov  sequence, 
t,  was  chosen  to  be  five  times  the  step  increment,  h.  This  second  set  of  numbers  will  be  denoted  by 
{ Vrj'k^i  ,  and  are  plotted  in  Figure  lb.  They  were  generated  from  {UrJic^i  by  the  difference 
equation, 

V,  =  , 

VK  =  Pv(l)VK-i  +  [l  -  pU  1)]1/2Uk,  2  <  K  <  100, 

where  pv(q)  =  exP  (~qh/r)  =  exp  (— q/5).  A  derivation  of  Equation  (2.6)  is  given  in  Reference  3. 
Finally,  f  UK  and  were  each  summed  to  obtain  the  r.w.’s  (Zn)  and  fYn}, respec¬ 

tively,  shown  in  Figure  2  and  Figure  3,  where  the  points  have  been  connected  by  straight  lines. 
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Figure  3 


In  addition  to  the  r.w.’s,  one  other  item  is  plotted  in  Figures  2  and  3.  Each  r.w.  was  fit  by  a 
first-degree  l.s.  polynomial,  and  the  line  segments  defined  by  the  fitted  polynomials  are  plotted. 
When  looking  at  these  figures,  it  should  be  remembered  that  both  r.w.’s  were  generated  by  under¬ 
lying  sequences  with  the  same  variance,  i.e.,  <?u  =  a\  =  1.0.  The  differences  between  the  two 
r.w.’s  result  from  the  fact  that  { VK >  is  correlated  and{UK)  is  not.  The  two  most  noticeable 
differences  are  that  {Yn}  is  much  smoother  than  {Zn},  and  the  variances  of  the  terms  of  { Yn)  are 
larger  than  those  of  the  corresponding  terms  of  { Znj. 

The  last  two  plots  in  this  section  illustrate  the  statistics  of  fZn}  and  { Yn).  Figure  4a  shows 
the  variance  of  the  terms  of  { Zn >  and  { Yn } ,  and  Figure  4b  gives  the  corresponding  standard 
deviations.  Figure  5  is  a  plot  of  pz(  1,  J),  pz{25,  J),  pz(50,  J)  and  pz(100,  J). 
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III.  FIRST-DEGREE  LEAST-SQUARES  FITS 


In  this  section,  only  first-degree  l.s.  polynomial  fits  to  r.w.  segments  are  considered.  This 
special  case  is  of  importance  for  two  reasons.  First,  the  simplicity  of  first-degree  fits  allows  the 
derivation  of  analytic  results  that  are  instructive  and  form  the  basis  for  the  generalization  to  fits  of 
arbitrary  degree.  Second,  many  practical  applications  use  first-degree  fits  to  r.w.  segments,  so  the 
development  of  the  fit  residual  statistics  without  resorting  to  matrix  notation  is  computationally 
useful. 

Our  immediate  objective  may  be  stated  as  follows.  Given  a  segment  of  the  r.w.  {Zn},  gen¬ 
erated  by  the  underlying  sequence  {UK},  express  the  residuals  of  the  first-degree  l.s.  polynomial  fit 
to  this  segment  as  linear  combinations  of  the  terms  of  {UK} .  Once  this  is  accomplished,  the  statis¬ 
tics  of  the  fit  residuals  may  readily  be  found  in  terms  of  the  statistics  of  (UK). 

Consider  the  r.w.  segment,  {Zn}  J^=1,  consisting  of  the  first  N  terms  of  {Zn}.  (It  will  be 
shown  later  in  this  section  that  no  generality  is  lost  by  treating  the  first  segment  of  length  N.) 

The  first  degree  l.s.  polynomial  fit  to  this  data  will  be 

=  +  ai^S*  tg  =  8h ,  8  =  1,  2,  ...,  N  (3.1 


where 


ao  = 


2(2N  +  1) 

N 

V  7  + 

6 

N(N  -  1) 

n  =  1 

hN(N  -  1) 

N 

I]  t*2"  > 

n  =  l 


(3.2 


and 


al 


6 

N 

V  7  + 

12 

hN(N  -  1) 

L  Z" 

n  =  l 

hzN(N2  -  1) 

N 

£  ‘A- 


(3.3 


n  =  1 


Zg  =  £  C(8,n)Zn 


Substituting  the  expressions  for  a0  and  ax  into  Equation  (3.1)  and  collecting  terms  yields 

N 

r 

n*l 

where 

C(8,  n)  =  c0[cj  +  c2(8  +  n)  +  c3n8] 


(3.4 


(3.5 


with 


C°  -  N(N  -  1)  ’  Cl  (2N  +  1),  c2  3  and  c3  =  ^  +  ^  . 

But,  Zn  is  just  the  sum  of  the  first  n  terms  of  (UK)  (by  Equation  (1.1)),  so  Equation  (3.4)  may  be 
written  in  the  form. 


N  n 

z«  -  £  £  C(«,n)UK. 

n«l  K-l 

This  double  sum  may  be  written  as  a  single  sum  by  introducing  a  new  set  of  weighting  coefficients. 
Thus, 

N 

Zg  =  £  D(«,K)Uk,  (3.6) 

K-l 

where 

N 

D(C,K)  =  £  C(8,j).  (3.7) 

j-K 

Substitution  of  Eq  uation  (3.5)  into  Equation  (3.7)  yields 

D(8,K)  =  d^  +d2«)  +  (d3  +  d48)K  +  (d5  +  d6£)K2J  (3.8) 

where 

d0  *  dj  =  (N  +  l)(N  +  2),  d2  *  -6 

d3  =  _(4N  +  5) ,  d4  =  d5  =  3,  d6  -  • 

Finally,  from  Equation  (1.3), 

rg  =  Zg  —  Zg ,  B  =  1,  2,  ...,  N. 


So,  using  Equations  (1.1)  and  (3.6)  the  fit  residuals  for  the  first  degree  l.s.  fit  become 

8  N 

r«  -  £  UK  -  £  D(£,K)Uk 
K-l  K-l 


or 

N 

rg  =  £  F(2,K)Uk,  (3.9) 

K-l 

where 

(  1  -  D(8,  K)  for  1  <  K  <  £ 

F(8,  K)  =  <  (3.10) 

[  -D(2,K)  for  2  <  K  <  N. 
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Thus,  by  introducing  a  third  set  of  weighting  coefficients,  the  F(£,  K)’s,  we  have  met  our  immediate 
objective.  Before  using  Equation  (3.9)  to  determine  the  statistics  of  the  fit  residuals,  several  facts 
concerning  the  three  sets  of  weighting  coefficients  should  be  pointed  out. 

From  the  definitions  of  C(£,  n),  D(£,  K),  and  F(£,  K)  given  by  Equations  (3.5),  (3.8)  and 
(3.10),  respectively,  the  following  facts  may  be  established: 


C(£,  n)  =  C(n,  £) 

(3.11) 

N 

C(£,  n)  =  C(N  - 

n  +  1,  N 

-  £  +  1) 

(3.12) 

£ 

n=l 

C(£,  n)  =  1 ,  £ 

=  1,  2, 

•  ••,  N 

(3.13) 

N 

D(£,  1)  =  1 ,  £ 

=  1,  2, 

...,  N 

(3.14) 

£ 

K  =  1 

D(£,  K)  =  £,  £ 

=  1,  2. 

•  --,  N 

(3.15) 

N 

F(£,  1)  =  0,  £ 

=  1,  2,  . 

....  N 

(3.16) 

£ 

K  =  1 

F(£,  K)  =  0 ,  £ 

=  1,  2,  . 

,.,N. 

(3.16) 

It  might  be  noted  that  the  computation  of  all  three  sets  of  weighting  coefficients  depends 
only  on  the  value  of  N,  the  number  of  terms  being  fit.  The  facts  presented  in  Equations  (3.11)— 
(3.16)  represent  only  a  few  of  the  results  and  relations  that  might  prove  useful  in  various  applica¬ 
tions,  but  they  should  serve  as  a  basis  for  further  derivations.  For  example,  if  we  think  of  the 
C(£,  n)’s  as  the  elements  of  an  N  X  N  matrix,  C,  then  Equations  (3.11)  and  (3.12)  show  that  C  is 
symmetric  about  both  of  its  diagonals.  Equation  (3.13)  implies  that  the  rows  of  C  all  sum  to  one, 
and  Equations  (3.11)  and  (3.13)  together  imply  that  the  columns  of  C  also  sum  to  one. 


The  C(£,  n)’s  are  particularly  useful  in  different  applications  since  they  define  the  relationship 
between  the  value  of  Zg  =  Z(tg)  for  a  first-degree  l.s.  polynomial  fit  and  the  data  values,  Zn  =  Z(tn), 
over  which  the  fit  was  performed.  The  only  assumption  made  during  the  derivation  of  Equa¬ 
tions  (3.4)  and  (3.5)  was  that  the  data  be  equally  spaced.  If  we  think  of  C(£,  n)  as  a  function  of  n 
for  some  fixed  value  of  £,  we  see  from  Equation  (3.5)  that  C(£,  1),  C(£,  2),  ...,  C(£,  N)  lie  on  a 
straight  line  whose  y-axis  intercept  is  CqIcj  +  c2£),  and  whose  slope  is  c0(c2  +  c3£).  The  constants 
c0,  Cj ,  c2,  and  c3  depend  only  on  N. 

A  A 

Similarly,  the  D(£,  K)’s  define  the  relationship  between  the  value  of  Zg  =  Z(tg)  for  a  first- 
degree  l.s.  polynomial  fit  and  the  values  {UK}  l  the  underlying  sequence  that  generated  the 
r.w.  segment  over  which  the  fit  was  performed.  In  this  instance,  we  see  from  Equation  (3.8)  that 
for  fixed  values  of  £,  the  weighting  coefficients  D(£,  1),  D(£,  2),  ...,  D(£,  N)  lie  on  a  quadratic 
curve.  Figures  6  and  7  are  plots  of  the  C(£,  n)’s  and  D(£,  K)’s,  respectively,  for  the  case  N  =  10, 
and  Figure  8  is  a  plot  of  the  associated  F(£,  K)’s  for  this  case.  For  the  sake  of  clarity,  Figure  8 
consists  of  two  plots,  Figures  8a  and  8b,  showing  F(l,  K)  through  F(5,  K)  and  F(6,  K)  through 
F(10,  K),  respectively. 
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Returning  to  our  main  theme,  we  see  from  Equation  (3.9)  that  the  mean  of  rg  is  zero,  since 
{UK}  is  a  zero  mean  sequence.  The  variance  of  rg  is  given  by 


o,(C)  =  E[rg]  =  ay 


N  N-l  N 

£  F2(C,  K)  +  2  £  £  F(M)F(C,j)pu(j-i) 

K=1  i=l  j=i+l 


(3.17) 


The  correlation  between  rt  and  rj,  I  <  J,  is  given  by  the  correlation  coefficient, 

Pr(I,J)  =  E[rirj]/or(I)or(J),  (3.18) 

where 

f  N 

Etrirj]  =  al  J  £  F(I,  K)F(J,  K) 

lK=i 

N  -I  N 

+  E  H  [F(I,i)F(J,j)  +  F(I,j)F(J,  i)lPu(j-i) 

i=l  j=i+l 


Since  Equation  (3.9)  equates  the  fit  residuals  to  a  linear  combination  of  the  UK’s,  it  follows 
that  the  residuals  are  gaussian  and  so  Equations  (3.17),  (3.18),  and  (3.19)  completely  specify  their 
statistics.  These  equations  represent  the  answer  to  one  of  the  primary  questions  addressed  by  this 
study. 

Figures  9, 10,  and  11  display  the  variances  of  the  fit  residuals  for  three  different  values  of  N. 
Each  of  the  plots  shows  the  results  for  (UK)  uncorrelated  and  for  (UK)  a  first-order  markov 
sequence  with  =  5.0.  In  all  cases,  =  1.0  and  h  =  1.0.  The  values  of  N  used  in  Figures  9, 10, 
and  11  are  10, 15,  and  50,  respectively.  Since  only  two  levels  of  correlation  of  the  underlying 
sequence  are  shown  in  these  plots,  it  is  not  possible  to  conclude  much  from  them  concerning  the 
effect  on  the  fit  residual  statistics  of  increasing  the  correlation  of  {UK}.  For  this  reason.  Figure  12 
has  been  included.  It  shows  the  effect  of  various  values  of  tv  on  o,  (1),  o?(4),  and  oj?(8),  with 
N  =  15,  ay  =  1.0,  and  h  =  1.0.  The  results  for  ry  =  0  (i.e.,  for  {UK}  uncorrelated)  through 
rgj  =  10.0  are  plotted;  then  a  break  in  the  plot  is  indicated,  followed  by  the  values  corresponding 
to  rj;  =  99.0  (The  fact  that  the  values  of  0^(2)  approach  zero  as  Ty  -*■  °°  is  easily  explained.  For 
very  large  values  of  tv,  {Uk}  looks  like  a  random  constant  sequence,  so  { Z is  almost  a  random 
ramp  sequence.  The  more  closely  { Zn}  resembles  a  ramp  the  better  it  will  be  approximated  by  a 
first-degree  polynomial.)  The  last  plot  associated  with  the  fit  residual  statistics  is  Figure  13.  It 
shows  correlation  coefficients  pr(I,  J)  for  I  =  1, 10,  20,  25,  and  J  =  1,  2,  ....  50,  for  the  case  N  =  50, 
of]  =  1.0,  h  =  1.0,  and  {UK}  uncorrelated. 

In  the  third  paragraph  of  this  section,  we  claimed  that  no  generality  is  lost  by  developing  the 
fit  residual  statistics  for  fits  to  r.w.  segments  of  length  N  by  treating  the  segment  {Zn) n«i  •  At  this 
point,  we  will  show  that  for  first-degree  l.s.  polynomial  fits  the  fit  residual  statistics  are  identical 
for  fits  to  arbitrary  segments  of  length  N  of  a  given  r.w.  (Our  usual  assumptions  concerning  {UK} 
still  obtain,  of  course.)  This  result  will  be  demonstrated  for  l.s.  polynomial  fits  of  arbitrary  degree 
in  the  next  section. 
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Figure  12 


Figure  1 3 


Let  {Zn}  be  an  r.w.  whose  underlying  sequence,  {UK},  is  a  stationary,  zero  mean  gaussian 
sequence.  Suppose  that  an  arbitrary  segment  of  length  N  has  been  chosen  from  {Zn}.  We  will 
denote  this  segment  by  {Zn}  The  terms  of  this  segment  are  given  by 

Zf+8  =  UK  =  z?  +  US+K  ’  8  =  2,  ...,  N. 

K  =  1  K"1 
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The  values  of  the  first  degree  l.s.  polynomial  fit  to  this  segment,  corresponding  to  the  times  t^+^, 
are, 


N  /  N  \  N 

z ^  =  £  C(C,n)Z|+n  =  D(e,K)U?+K  +  Z*  C(S,n). 

n  =  1  \K  =  1  /  n  =  1 


But,  from  Equation  (3.13), 


N 

2]  C(8,  n)  =  1,  for  E  =  1,  2,  ...,  N. 

n  =  1 


Thus, 


=  Zf 


N 


+  2]  D(8.K)U|^k 

K  =  1 


and  so,  the  Cth  fit  residual  is 


-  z{+8  =  2J 

K>1 


F(C,  K)U?+k. 


(3.20) 


Since  {UK }  is  stationary,  r ^  is  zero  mean  and  the  variance  and  correlations  are  given  by  Equa¬ 
tions  (3.17),  (3.18),  and  (3.19).  This  completes  the  demonstration. 

Now,  the  second  of  our  primary  questions  concerns  the  relationship  between  the  fit  residuals 
of  fits  to  different  segments  of  the  same  r.w.  Given  our  usual  restrictions  on  {UK},  suppose  that 
two  nonoverlapping  segments  of  length  N  of  the  r.w.  (Zn)  have  been  fit  by  first-degree  l.s.  poly¬ 
nomials.  To  be  specific,  let  )Zn}^^+1  and  { Z„ }  ntj+i ,  with  £  >  t?  +  N,  be  the  first  and  second 
segments,  respectively.  Then  by  the  argument  leading  to  Equation  (3.20)  we  find  that  the  Cth 
residuals  of  the  fits  to  the  first  and  second  segments  are  given  by 


and 


4l)  =  £  F(C,K)U„+k 

K  =  1 


(3.21) 


r 


(2) 

V 


N 


E 

K  =  1 


F(«,  K)U^+k, 


15 


respectively.  We  have  just  shown  that  the  statistics  for  these  two  sets  of  residuals  are  identical,  so 
the  only  remaining  question  concerns  the  correlation  between  a  residual  of  the  first  fit  and  one  of 
the  second  fit. 

From  Equations  (3.21),  the  correlation  between  rj1*  and  r*j2*  is  given  by  the  correlation  coef¬ 
ficient,  (I  <  J), 


where 


(l,2)/r  T\ 

Pi  U>J) 


=  E 


n-r'I-']/ar]Orj, 


r(Dr(2)l 

rl  rJ 


(3.22) 


4l,42) 


N 


L 


L 


i  =  i 


F(I,  i)F(J,  j)pu($  ~  i?  +  j  -  i) . 


(3.23) 


The  standard  deviations  in  Equation  (3.22)  are  found  from  Equation  (3.17)  and  do  not  depend 
upon  which  set  of  residuals  are  under  consideration.  It  is  clear  that  if  {UK}  is  a  white  sequence 
then  the  correlation  between  any  residual  from  the  first  fit  and  any  residual  from  the  second  fit 
will  be  zero. 

The  third,  and  last,  of  the  primary  questions  mentioned  in  section  I  asks  about  the  statistics 
of  the  fitted  coefficients,  a0  and  at.  The  statistics  of  a0  represent  the  first  occurence  in  this  report 
of  statistics  that  depend  upon  where  the  fitted  segment  occurs  in  {Zn}.  We  will  begin  by  deriving 
the  statistics  for  these  coefficients  when  the  first  N  terms  of  {Zn}  constitute  the  fitted  segment 
and  then  will  show  how  the  statistics  for  a0  are  computed  for  an  arbitrary  segment. 


Consider  the  segment,  {Zn}JjLj ,  of  the  r.w.  {Zn}  whose  underlying  sequence  is  zero  mean, 
gaussian,  and  stationary.  Using  Equation  (1.1)  in  Equations  (3.2)  and  (3.3),  we  may  write  a0  and 
a!  in  terms  of  {UK},  for  N  >  2; 


where 


and 


a0 

al 

G0(K) 

Gt(K) 


N 


£  G0(K)Uk, 

K  =  1 


£  GjWUk, 


K  =  1 


_l _ 

N(N-l) 


-6 _ 

hN(N2  -1) 


3K2 

K2 


(4N  +  5)K  +  (N2  +  3N  +  2) 


(N  +  2)K  +  (N+  1) 


(3.23) 


(3.24) 


(3.25) 


(3.26) 


It  follows  from  Equations / 3.23)  and  (3.24)  that  a0  and  aj  are  both  zero  mean  and  gaussian  and 
that  their  variances  are  given  by 
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<(N)  =  E[a2]  =  al 


N  N-l  N 

2]  G2(K)  +  2^  2]  Gq(i)Gq(j)Pu(j-i) 

K  =  1  i  =  l  j  =  i  +  l 


(3.27) 


where  q  =  0,  1.  The  correlation  between  a0  and  ax  is  given  by  the  correlation  coefficient, 

Paoai  =  E[a0ai]/<Vai  (3.28) 


where 


E[aoai]  =  al  1  £  G0(K)Gx(K) 


N 

L 

K  =  1 

N-l  N 


+  2]  11  tG0(i)G1(j)  +  G0(j)G1(i)]pu(j  -  i) 

i“l  j  =  i  + 1  j 


(3.29) 


From  the  definitions  of  G0(K)  and  GX(K)  given  in  Equations  (3.25)  and  (3.26),  respectively, 
the  following  facts  may  be  established.  For  N  =  2,  3,  ..., 


G0(l)  =  1, 

(3.30) 

N 

2]  G0(K)  =  0, 

K  =  1 

(3.31) 

N 

2]  Gg(K)  =  [(2N  +  1)(N  +  1)(N  +  2)]/[15N(N  -  1)] , 

K=  1 

(3.32) 

Gx(l)  =  0, 

(3.33) 

N 

2]  GX(K)  =  1/h, 

K  =  1 

(3.34) 

N 

22  G|(K)  =  [6(N2  +  l)]/[h25N(N2  -  1)] . 

(3.35) 

K=  1 


For  white  underlying  sequences  the  double  sum  in  Equation  (3.27)  vanishes,  so  for  cases  where 
<UKj  is  white, 
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<(N)  = 

and 


~  6(N2  +  1)  I  o2 

. h25N(N2  -  1)  °U  ' 

In  Figures  14  and  15  we  have  plotted  of0  and  °lv  respectively,  for  the  case  of  a  white  underlying 
sequence  with  unit  variance  and  time  step  h  =  1,  for  2  <  N  <  25. 

Now,  let’s  consider  the  arbitrary  segment,  (Zn)  of  length  N.  We  may  define  a  new 

segment,  {X,,}  ^=1,  where 

Xn=Z|+n-Z?,  n  =  1,  2,  ...,  N ;  (3.36) 


(2N  +  l)(N  +  l)(N  +  2) 


15N(N  - 1) 


°u 


i 


so 


Xn  =  E  U?+K  •  (3-37) 

K  =  1 

Since  Z?+n  =  Z^  +  X„,  it  is  easy  to  see  (from  Equations  (3.2)  and  (3.3))  that  the  first-degree  l.s. 
polynomial  fit  to  the  segment  {Zn}j£|+1  is  given  by  Z?  plus  the  first-degree  fit  to  {X„}£L , .  That 
is,  the  desired  fit  is  given  by 

Z?+s  =  (Z?  +  a0)  +  aite,  te  =  Eh,  E-l,  2,  ...,  N,  (3.38) 

where 

a0  =  t  Go(K)U?+k 
k=i 

ai  =  E  G^KJU^k. 
k  =  i 


The  sequence  (UK)  is  stationary  so  a0  and  ax  are  zero  mean  and  their  variances  are  given  bv 
Equation  (3.27). 

The  constant  term  in  Equation  (3.38),  i.e.,  the  constant  term  in  the  first-degree  fit  to 
Zn'n=£+i,  has  a  variance  given  by 

o?o  =  E[(Z?  +  a0)2}  =  E[Zf)  +  2E[Z?a0]  +  E[ag)  .  (3.39) 
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Figure  15 


The  expectation  E[Z|]  is  given  by  Equation  (2.4),  E[ag]  is  given  by  Equation  (3.27)  as  mentioned 
above,  and 


E[Z|a0] 


L 


j-i 


Go(i)Pu(£  +  j-0 


(3.40 


For  white  underlying  sequences,  the  right-hand  side  of  Equation  (3.40)  vanishes. 


Similarly,  the  correlation  between  ax  and  the  constant  term  in  Equation  (3.38)  is  given  by  the 
correlation  coefficient, 

=  E[(Z£  +ao)all/3a0aa1  (3-41) 

where 

^[(Z^+aoJaj]  =  EfZ^a^]  +  E[a0a1],  (3.42) 

Finally,  the  expectation  Eta^  ]  is  given  by  Equation  (3.29)  and 


EtZ^aj]  -  Oy 


%  N 

£  £  Gi(i)Pu(€  +  i-i) 


i=l  j=l 


(3.43) 


IV.  LEAST-SQUARES  FITS  OF  ARBITRARY  DEGREE 


In  this  section,  we  generalize  the  results  given  in  section  III  in  three  ways.  The  principal 
generalization  is  that  polynomial  fits  of  arbitrary  degree  are  allowed.  In  addition,  the  independent 
variable  will  be  (t  — 10)  instead  of  t,  where  t0  is  arbitrary.  Thus,  the  fitted  polynomials  will  be  of 
the  form, 

d 

W  =  £  aj(t-t0)j.  (4.1) 

i=o 


Finally,  the  terms  of  {UK}  and  {Zn}  will  not  be  assumed  to  be  evenly  spaced  in  lime. 


These  generalizations  are  easily  accomodated  by  the  introduction  of  matrix  notation.  As  in 
section  III,  we  will  begin  by  considering  fits  to  the  r.w.  segment,  {Zn}£Lj,  consisting  of  the  first 
N  terms  of  {Zn},  and  show  later  that  no  generality  is  lost  by  this  choice.  As  usual,  we  will  assume 
that  the  underjying  sequence,  {UK},  is  a  stationary,  zero  mean  gaussian  sequence.  The  (column) 
vectors  U,  Z,  Z,  r  and  a  are  defined  by 


[Uj,  u2, 

[Zx  z2, . 

zn]t 

[Zj,  z2, . 

7  1T 
...,  z-Nj 

lri>  r2,  .. 

•,rN]T 

[a0>  ai  i  •  ■ 

•>ad]T 

where  Z^  =  P{j(tg),  r^  =  Z$  ~Zq,  and  aj,  i  =  0,  1,  ...,  d  are  the  coefficients  of  the  l.s.  polynomial 
Pd(t).  We  will  always  assume  that  N  >  (d  +  1). 


The  c/th-degree  l.s.  polynomial  fit  to  Z  is  given  by 

a  =  (ATA)_1ATZ, 

where  A  is  the  N  X  (d  +  1)  observation  matrix.  For  this  case,  A  is  simply 

1  ( t-o )  •  -  ■  (^i  -  to)d 


(4-2) 


A  = 


1  (f-2  ^o)  •  •  •  (^2  to)d 


1  (tN  ~  t0)  ...  (tN  t0)d 


(4.3) 


The  values  of  the  fitted  polynomial  and  the  associated  residuals  at  the  times  tx,  t2,  ....  tN,  are 
given  by 

Z  =  CZ  (4.4) 

and 

r  =  HZ  (4.5) 

where  C  and  H  are  the  N  X  N  matrices  defined  by 

C  =  A(AtA)_1At,  (4.6) 

H  =  (I  —  C) ,  (4.7) 

with  I  denoting  the  N  X  N  identity  matrix. 

Now,  since  Zn  is  just  the  sum  of  the  terms  UK,  K  =  1,  2,  ...,  N,  Z  and  U  are  related  by 

Z  =  SU  (4.8) 

where  S  is  the  lower  triangular  N  X  N  matrix  whose  elements  on  and  below  the  main  diagonal  are 
all  unity.  Thus,  using  Equation  (4.8)  in  Equations  (4.4)  and  (4.5)  we  have 

Z  =  DU  (4.9) 

and 

r  =  FU,  (4.10) 

where  D  and  F  are  the  N  X  N  matrices  defined  by 
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D  =  CS, 


(4.11) 


F  =  HS.  (4.12) 

Since  {UK}  is  zero  mean  and  gaussian,  it  follows  that  r  is  also  zero  mean  and  gaussian.  Thus, 
the  statistics  of  r  are  completely  specified  by  its  covariance  matrix,  Qr.  From  Equation  (4.10), 

Qr  =  E[rrTl  =  FQuFt,  (4.13) 

where  Qy  =  E[  UUT]  is  the  covariance  matrix  of  U. 

At  this  point,  we  have  determined  the  statistics  of  the  fit  residuals  for  fits  to  the  segment 
.  We  will  now  show  that,  for  any  other  segment  of  length  N,  whose  spacing  scheme  is  iden¬ 
tical  to  that  of  {Zn} £Lj,  the  fit  residual  statistics  will  also  be  given  by  Equation  (4.13).  The  proof 
of  this  claim  will  make  use  of  the  fact  that  the  rows  of  C  all  sum  to  one.  So,  we  need  the  following 
lemma. 

Lemma  (1):  Let  A  be  the  N  X  (d  +  1)  observation  matrix,  given  in  Equation  (4.3),  associated 
with  a  dth-degree  l.s.  polynomial  fit  to  N  points,  and  let  C  be  the  N  X  N  matrix  defined  by 
C  =  A(ATA)-1AT.  Then  the  rows  of  C  each  sum  to  one. 

Proof  of  Lemma  (1):  Consider  the  matrix  H  =  (I  -  C).  The  product  ATH  vanishes  because 

AtH  =  AT(I-C)  =  AT(I-A(ATA)-1AT)  =  AT  -  AT  =  0. 

By  Equation  (4.3),  the  first  row  of  AT  consists  of  N  ones,  so  it  follows  from  the  fact  that  ATH  =  0 
that 

N 

2^  Hjj  =  0,  for  j  =  1,  2 . N. 

i  =1 

Since  H  is  symmetric,  this  implies  that 

N 

J  '  Htj  =  0,  for  i  =  1,  2,  ...,  N. 
i  =  i 

Now,  from  the  definition  of  H,  C  =  I  -  H,  so 
N  N 

E  CU  -  1  -  £  Hij  =  1,  for  i  =  1,  2,  ...,  N, 
j*i  j=i 

i.e.,  the  sum  of  each  row  of  C  is  one.  Q.E.D. 

It  should  be  noted  that  if  A  is  an  arbitrary  observation  matrix  associated  with  some  l.s.  proce¬ 
dure,  the  rows  of  A(ATA)_1  AT  will  not,  in  general,  sum  to  one.  For  example,  consider  the  4X2 
matrix, 
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A  = 


1 

2 


2  2 
3  1 


For  this  case,  ATA  is  nonsingular  and  the  four  rows  of  A(ATA)-1  AT  sum  to  21/29,  38/29,  30/29, 
and  17/29,  for  i  =  1,  2,  3  and  4,  respectively. 

With  the  preceding  lemma  in  hand,  we  may  now  prove  the  following  theorem. 

Theorem:  Let  {Zn}  be  a  r.w.  whose  underlying  sequence,  {UK},  is  zero  mean,  stationary, 
and  gaussian.  Consider  the  two  segments  of  {Zn}  denoted  by 


and 


Zj  _  [Zj,  Z2,  ....  ZN] 

zu  =  [Z{+1,Z?+2,...,Zf+N]T, 


where  £  is  an  arbitrary  integer  such  that  £  >  N.  If  the  spacing  schemes  for  the  two  segments  are 
identical;  i.e.,  if  (ti  +  1  -  tj)  =  (t^+j+j  -tt+i)fori  =  1,  2,  ...,  N  -  l,and  each  segment  is  approxi¬ 
mated  by  a  dth-degree  l.s.  polynomial,  then  the  fit  residuals  for  the  two  fits  will  have  identical 
statistics,  given  by  Equation  (4.13). 

Proof  of  Theorem:  Let  Uj  =  (Ult  U2,  ...,  UN]T  and  Un  =  [U^+i,  U|+2,  •••>  u£+n1T’  with 
associated  covariance  matrices  QU(  and  respectively.  The  elements  of  Zu  are  given  by 

£+«  £+e 

z|+e  =  £  UK  =  h  +  E  UK  ’  f°r  8  =  1,  2 . N. 

K=  1  K-f+1 

Thus,  Zn  may  be  written  in  the  form, 

Zn  =  SUU  +  Z*[l,  i,...,  i]T. 

The  fit  values,  Zn,  are  given  by, 

Z„  =  CZ„  =  CSU„  +  ZtC[l,  1,  ....  1]T 

=  DUn  +  Z{C[1,  1 . 1]T 

=  DUU  +  Z^[  1,  1,  ....  1]T, 

where  the  last  equality  follows  because  the  rows  of  C  all  sum  to  one.  The  fit  residuals  for  the  fit 
to  Zjj  are  then 

fII  =  Zjj  ~  Zjj  =  SUjj  -  DUjj  =  FUjj  . 
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7*  * 


Clearly,  rn  is  zero  mean  and  gaussian,  with  covariance  matrix, 

Q,„  -  FQu/T. 

By  Equation  (4.13),  the  fit  residuals  for  the  fit  to  Zt  have  covariance  matrix, 

Qr,  =  FQu  Ft  . 

Since  {UK}  is  stationary  and  Uj  and  Uu  have  identical  spacing  schemes,  by  assumption,  it  follows 
that  Qu  =  Qjji  and  hence  =  Q  .  Q.E.D. 

We  have  just  shown  that  if  the  conditions  stated  in  the  preceding  theorem  are  met  then  the 
individual  statistics  for  rj  and  rn  are  identical.  To  complete  our  discussion  of  the  relationship 
between  the  two  sets  of  residuals  we  need  to  determine  the  cross-covariance  matrix,  Rj  u,  for  rj 
and  ru.  Since  rj  =  FUj  and  rn  =  FUn,  both  are  zero  mean  and  Rj  D  is  given  by 

Rl,n  =  Ef'i'nJ  =  FE[Uru7iJFT .  (4.14) 


If  (UK }  is  a  white  sequence,  it  follows  from  Equation  (4.14)  and  the  fact  that  (UK)  is  gaussian 
that  rj  and  rn  are  statistically  independent  for  this  case. 

Finally,  we  need  to  address  the  question  of  the  statistics  of  the  fit  coefficients,  a.  As  men¬ 
tioned  in  section  III,  the  statistics  of  the  constant  term,  a0,  depend  on  where  the  fitted  r.w.  segment 
occurs  in  {Zn}.  We  will  begin,  as  before,  by  deriving  the  statistics  of  a  for  the  segment  consisting 
of  the  first  N  terms  of  {Zn}  and  then  will  show  how  the  statistics  are  computed  for  an  arbitrary 
segment.  The  extension  will  require  the  result  given  in  the  following  lemma. 

Lemma  (2):  Let  A  be  the  observation  matrix  associated  with  a  l.s.  rfth  degree  polynomial  fit 
to  the  N  data  points,  Zn  =  Z(tn),  n  =  1,  2,  ...,  N,  given  explicitly  by  Equation  (4.3).  Let 
W  be  the  (d  +  1)  X  N  matrix  given  by  W  =  (A1  A)-1  AT,  V  be  an  N-dimensional  column  vector  of 
ones,  and  e  be  the  (d  +  l)-dimensional  column  vector  whose  first  component  is  one  and  whose  re¬ 
maining  components  are  all  zero;  i.e.,  V  =  [  1,  1,  ...,  1  ]T  and  e  =  [  1,  0,  0,  ...,  0]T.  Then,  WV  =  e. 

Proof  of  Lemma  (2):  By  hypothesis  (see  Equation  (4.2)),  W  acts  on  a  given  set  of  N 
points  to  yield  the  coefficients  of  the  unique  dth-degree  polynomial  which  minimizes  rTr,  where 
r  is  the  column  vector  of  fit  residuals.  Thus,  WV  will  be  the  coefficients  of  the  dth-degree  polyno¬ 
mial  which  best  fits  (in  the  l.s.  sense)  the  components  of  V.  If  WV  =  e,  then  r  =  V  -  Ae  =  0  and 
rTr  =  0.  But  rTr  >  0  for  any  r,  so  it  follows  that  no  other  (d  +  1)- dimensional  column  vector, 
e  +  e,  could  yield  a  lower  value  for  rTr.  Since  the  best  fitting  l.s.  cfth-degree  polynomial  is  unique, 
WV  must  equal  e.  Q.E.D. 

Consider  the  segment  fZn}  „=1  of  the  r.w.  {Zn}  whose  underlying  sequence,  (UK },  is  zero 
mean,  stationary,  and  gaussian.  From  Equations  (4.2)  and  (4.8)  we  have 


a  =  GU, 


(4.15) 
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G  =  (AtA)'1AtS. 


(4.: 


It  follows  from  Equation  (4.15)  that  a  is  zero  mean  and  gaussian,  and  that  its  covariance  matrix 
is  just 


Qa  =  GQuG1  . 


(4. 


Now,  let’s  consider  the  arbitrary  segment,  {Zn}|t,jj+1,  of  length  N.  Proceeding  as  we  did  in 
section  III,  we  define  a  new  segment,  {X„}  „=1 ,  where  (see  Equation  (3.36)) 


Xn  Z| ,  n  1,  2,  ...,  N 


or 


=  £  U?+K. 

K  =  1 

Letting  Z  =  [Z^+1 ,  Z%+2,  ...,  Z^+N]T  and 

X  =  [Xl5X2 . Xn]t 

we  see  from  the  definition  of  Xn  given  above  that  these  vectors  are  related  by 

X  =  z  -  ztv 

or 

z  =  X  +  ztv, 

where  V  is  an  N-dimensional  column  vector  each  of  whose  components  is  one,  i.e.,  V  =  [  1,  1,  ...,  1] 


The  fits  to  X  and  Z  are  thus  given  by 


ax  =  (AtA)-1AtX  =  WX  (4.1 

and 

az  =  (AtA)‘1AtZ  =  WZ  =  WX  +  Z|WV  =  ax  +  Z^WV. 

But,  from  Lemma  (2)  we  know  that  WV  =  e,  so 


az  ~  ax  +  ZsB. 


(4. 


Thus,  the  covariance  matrix  of  the  coefficients  of  the  fit  to  Z  is  just, 

Qaz  =  E[aza|]  =  Qax  +  E[axeTZ^j  +  E[Z?ea£]  +  E[ZfeeTl ,  (4.20) 

where  Qa^  is  the  covariance  matrix  of  ax  • 

Since  X  =  SUX,  where  Ux  =  [U^+1,  U^+2,  ...,  U^+N]T  and  Z^  is  simply  the  sum  of  Uj 
through  Us,  it  follows  that  Q,  is  given  by 

s  “X 

Qax  =  GE[UXUX]GT  =  GQUxGt, 

and  the  other  terms  of  Equation  (4.20)  are  also  easily  rewritten  in  terms  of  the  statistics  of  {UK } . 

It  should  be  noted  that  the  second,  third,  and  fourth  terms  on  the  right  of  Equation  (4.20)  act  to 
modify  only  the  first  row  and  first  column  of  Q3x . 

Figures  16  and  17  display  the  variances,  of(£),  of  the  fit  residuals  for  fits  of  degree  d  =  1,  2,  5 
and  8.  In  each  case,  N  =  50,  Oy  =  1.0,  and  (tK+1  -  tK)  =  1.0,  for  K  =  1,  2,  ....  The  results  shown 
in  Figure  16  are  for  the  case  with  {UK}  a  white  sequence.  Figure  17  shows  the  corresponding  re¬ 
sults  with  {UK}  a  first-order  markov  sequence  whose  correlation  time,  ru,  is  equal  to  5.0.  (Samples 
of  the  underlying  sequences  and  the  associated  random  walks  for  these  two  cases  were  presented  in 
Figures  1,  2,  and  3.)  It  should  be  noted  that  the  vertical  scales  for  the  plots  with  d  =  1  and  d  =  2 
differ  between  Figures  16  and  17,  while  the  plots  with  d  =  5  and  d  =  8  have  the  same  vertical  scales 
in  both  figures. 

It  is  interesting  to  note  that  the  values  of  of  (8)  with  d  =  8  are  slightly  higher,  for  most  values 
of  C,  when  {UK}  is  white  than  they  are  for  the  correlated  case.  This  is  due  to  the  fact  that  an  r.w. 
generated  by  a  white  underlying  sequence  contains  more  power  at  higher  frequences  than  one 
generated  by  a  correlated  underlying  sequence.  This  is  illustrated  by  a  comparison  of  Figures  2 
and  3. 


V.  EXAMPLE;  ATOMIC  CLOCK  ERRORS 


A  clock  is  any  device  which  counts  the  cycles  of  a  periodic  phenomenon.  Among  the  most 
stable  clocks  in  use  are  the  atomic  clocks  which  form  the  basis  for  atomic  time  scales  such  as  Inter¬ 
national  Atomic  Time  (TAI).  Atomic  time  is  used  primarily  as  a  measure  of  time  interval  and  is 
based  on  the  electromagnetic  oscillations  produced  by  quantum  transitions  within  the  atom. 

If  an  atomic  clock  is  used  as  a  component  of  some  distance  measurement  system,  then  clock 
errors  become  part  of  the  total  measurement  error  of  the  system.  When  data  analysis  is  performed 
on  the  output  of  such  a  system,  it  is  often  convenient  to  model  (approximate)  the  clock  errors 
over  some  time  interval  by  a  polynomial.  In  this  way,  a  substantial  portion  of  the  clock’s  contribu¬ 
tion  to  the  total  measurement  error  may  be  removed.  This  is  the  motivation  for  the  work  presented 
in  this  example. 
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1  25  50  ,  25  60 


I'  I 

Figure  16  Figure  17 

Atomic  clocks  are  subject  to  both  deterministic  and  random  errors  causing  deviations  from  an 
idealized  time  scale.  The  random  time  error,  X(t),  results  from  the  summation  of  all  prior  time 
changes  induced  by  random  changes  in  frequency,  known  as  fractional  frequency  fluctuations, 
y(t).  These  are  defined  by 

y(t)  =  AF(t)/F, 

where  F  is  the  ideal  frequency  of  the  clock’s  oscillator  and  AF(t)  represents  the  departure  of  the 
actual  frequency  from  F.  Standard  procedures  for  specifying  the  statistics  of  y(t)  involve  the  use 
of  the  Allan  variance  or  the  use  of  power  spectral  densities  (see  Chapter  8  of  Reference  4). 
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Since  the  random  time  error  is  the  sum  of  all  prior  random  time  errors  caused  by  the  random 
frequency  fluctuations,  the  statistics  of  the  random  time  error  may  be  developed  from  the  statistics 
of  y(t).  In  the  present  example,  we  have  assumed  that  measurements  of  X(t)  have  been  made 
discretely  with  a  fixed-time  interval,  At,  between  measurements  and  that  y(t)  may  also  be  approxi¬ 
mated  as  a  discrete  sequence.  Thus,  in  terms  of  our  previously  adopted  notation,  {yK}  plays  the 
role  of  {UK}  and  {XjJ  the  role  of  {Zn}.  Given  that  the  error  in  the  clock  at  time  t0  is  zero  (i.e., 
that  t0  corresponds  to  a  reset  or  calibration  time  of  the  clock)  then  {X,,}  takes  the  form  of  a 
random  walk,  with 


-  At 


L 


K-l 


VK- 


Figure  18  is  a  plot  of  the  square  root  of  the  Allan  variance  for  a  typical  cesium  clock.  Using 
this  information,  a  sequence  of  random  time  errors  was  generated  whose  Allan  variance  closely 
approximates  that  shown  in  the  figure.  These  time  errors  were  converted  to  range  or  distance 
errors  by  multiplying  by  the  speed  of  light.  The  plot  of  these  errors  is  labeled  “actual  range  error” 
in  Figure  19.  The  data  are  given  over  a  five-day  period  with  the  clock  error  assumed  to  be  zero 
initially.  Notice  that  the  error  appears  quite  systematic  over  the  five-day  span,  even  though  it  does 
represent  a  segment  of  a  random  walk.  This  is  a  result  of  the  long  correlation  time  of  the  under¬ 
lying  sequence,  so  the  horizontal  scale  would  have  to  be  extended  over  a  much  longer  time  period 
in  order  to  see  a  pattern  more  representative  of  the  behavior  we  expect  to  see  from  a  random  walk 
process. 


Figure  18 
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Figure  19 


A  second-degree  polynomial  of  the  form, 

P2(t)  =  a0  +  ajt  +  a2t2  , 

was  fit  to  the  range  errors  over  the  five-day  period.  The  resulting  coefficient  values  were 

a0  =  1.0589  m 

aj  =  0.39213  X  10‘4  m/sec 

a2  =  -0.12103  X  10~10m/sec2. 

The  associated  fit  residuals  are  plotted  in  Figure  19.  The  contributions  of  the  linear  and  quadratic 
terms  at  t  =  0.432  X  106  sec  (=  5  days)  are  16.94  m  and  -2.26  m,  respectively,  so  for  time  intervals 
of  much  shorter  duration  than  five  days  a  first-degree  polynomial  should  be  adequate. 

For  example,  an  eight-hour  span  of  this  simulated  cesium  clock  error  was  fit  with  a 
polynomial  of  the  form, 

Pj(t)  =  ag  +  a^t. 

The  fit  residuals  for  this  case  are  shown  in  Figure  20.  The  theoretical  standard  deviations  of  the 
residuals  for  a  first -degree  fit  over  eight  hours  are  given  in  Figure  21. 

Finally,  Figure  22  gives  a  contour  plot  of  the  matrix  of  correlation  coefficients  of  the  fit 
residuals  for  the  linear  model  over  the  eight  hour  span.  Notice  that  the  correlation  between 
residuals  separated  by  a  fixed  time  interval  varies  with  the  location  of  the  residuals  within  the  fit 
span.  This,  as  well  as  the  curve  in  Figure  21,  demonstrates  the  non-stationarity  of  the  statistics  of 
the  residuals. 
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Figure  20 


Figure  21 


*  *  * 

This  concludes  the  present  report.  Clearly,  the  practical  application  of  l.s.  polynomial  approxi¬ 
mations  to  one  dimensional  r.w.  segments  raises  many  questions  that  were  not  addressed  in  this 
report.  It  is  hoped,  however,  that  the  results  and  techniques  that  have  been  presented  will  serve  as 
a  basis  for  the  derivations  required  to  answer  specific  questions  of  interest  to  the  reader. 
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Figure  22 
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